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PREFACE 


This  paper  reports  the  results  of  work  completed  under  Contract  # 
F3361 5-89-C'0009,  awarded  in  response  to  a  Phase  2  SBIR  proposal 
titled  "Development  of  the  BFITS  Intelligent  Tutoring  System."  This  is  the 
third  and  final  report  relating  to  this  overall  effort.  A  prototype  system 
was  developed  during  the  Phase  1  effort  which  became  known  as  the 
"Basic  Flight  Instruction  Tutoring  System,"  or  BFITS  for  short.  Prior  to 
Phase  2  of  this  effort,  an  interim  effort  was  performed  which  primarily 
focused  on  specification  of  systems  and  courseware  to  be  developed 
during  Phase  2  of  the  BFITS  development  effort. 
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DEVELOPMENT  OF  THE  BASIC  FLIGHT  INSTRUCTION 
TUTORING  SYSTEM  (BFITS) 


SUMMARY 

The  Basic  Flight  Instruction  Tutoring  System  (BFITS)  was  de¬ 
veloped  to  support  the  research  mission  of  the  Learning  Abilities  Meas¬ 
urement  Program  (LAMP).  Toward  that  end,  BFITS  teaches  both  the 
what  and  how  of  basic  flight,  while  monitoring,  tracking,  and  recording 
the  student's  behavior  as  he/she  works  with  the  tutor.  The  results  of 
studies  using  BFITS  will  enable  LAMP  researchers  to  make  informed 
recommendations  leading  to  improved  personnel  selection  and  classifica¬ 
tion  methods  for  the  Air  Force. 

BFITS  consists  of  an  instructional  module,  a  flight  simulator,  and 
a  performance  evaluator.  Each  of  these  modules  work  together  to  involve 
the  student  in  tasks  requiring  both  cognitive  and  psychomotor 
(hand-eye-foot  coordination)  skills.  The  instructional  module  teaches  the 
what  of  basic  flight  using  text,  graphics,  and  animation.  The  flight 
simulator  provides  the  student  with  a  hands  on  experience  of  the  how  of 
flying  a  small  airplane.  It  is  used  in  conjunction  with  a  number  of  easily 
changed  flight  scenarios  that  direct  the  student's  current  task  and  pro¬ 
vide  performance  evaluation  criteria  and  hint  messages.  The  perform¬ 
ance  evaluator  tracks  student  progress  and  allows  the  student  to  view  a 
graphical  display  of  his/her  performance  as  measured  against  the  evalu¬ 
ation  criteria.  It  also  provides  the  student  with  the  option  of  viewing  the 
just  completed  flight,  and  with  review  messages  indicating  the  areas 
where  the  evaluation  criteria  were  exceeded. 

BFITS  is  a  powerful  and  flexible  research  tool  with  great  potential. 
Its  use  will  determine  its  validity  as  an  indicator  of  learning  ability. 
Future  deveiopments  could  include  voice  feedback,  diagnostic  feedback 
after  each  practice  flight,  and  more  powerful  authoring  tools.  BFITS 
could  also  be  used  in  new  applications  such  as  assessing  the  effects  of 
drugs,  stress,  or  other  environmental  factors  on  pilot  performance. 


INTRODUCTION 
LAMP  and  BFITS 

In  this  report,  we  describe  the  development  of  the  Basic  Flight 
Instruction  Tutoring  System  (BFITS).  BFITS  was  developed  in  support  of 
the  research  mission  of  the  Learning  Abilities  Measurement  Program 
(LAMP).  The  mission  of  LAMP  is  (a)  to  identify  through  basic  research  the 
cognitive  abilities  that  enable  students  to  acquire  knowledge  and  skills, 
and  (b)  develop  and  validate  computer-based  tests  of  these  abiiities.  The 
ultimate  goal  is  to  use  this  computer-based  cognitive  ability  measure¬ 
ment  (CAM)  technology  to  improve  personnel  selection  and  classification 
in  the  Air  Force. 

LAMP  has  two  major  activities.  One  is  basic  research  on  learning 
and  cognition  which  leads  to  the  development  of  CAM  tests.  The  second 
is  the  validation  of  these  CAM  tests  as  predictors  of  complex  skill  acquisi¬ 
tion.  To  accomplish  this  second  goal,  LAMP  researchers  have  developed 
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computer-based  learning  environments  designed  for  complex  learning 
assessment  (CLASS).  Learning  environments  have  been  developed  to 
deliver  instruction  in  Pascal  programming  (Shute  &  Kylionen,  1990), 
basic  electricity  (Shute,  1990),  and  flight  engineering.  In  each  case,  the 
learning  environment  has  been  designed  to  record  various  learning 
measurements  as  the  student  progresses  through  the  course  and  to 
assess  learning  outcomes  at  the  end  of  instruction. 

BFITS  was  designed  with  these  same  goals  in  mind.  It  provides  a 
means  for  observing  and  tracking  the  behavior  of  students  as  they 
attempt  to  learn  basic  flight  procedures.  Designed  to  teach  both  the 
what  and  the  how  of  basic  flight,  BFITS  also  records  data  during  (a)  the 
initial  presentation  of  concepts  and  principles,  and  (b)  the  practice  phase 
in  which  students  apply  and  refine  their  knowledge  in  flight  training 
sessions  using  the  BFITS  simulator.  BFITS  will  provide  a  special  benefit 
in  that  it  will  link  LAMP  to  the  aircrew  selection  and  classification  func¬ 
tion  within  the  Armstrong  Laboratory.  As  a  result  of  studies  with  BFITS, 
LAMP  researchers  will  be  able  to  make  informed  recommendations  for 
new  tests  to  be  included  in  a  future  generation  of  the  Basic  Attributes 
Test  used  for  aircrew  selection  and  classification. 


Bgckgrogpd 

Research  and  development  on  pilot  selection  and  training  began  to 
receive  emphasis  during  World  War  II  (Koonce,  1984).  One  concern 
motivating  training  research  was  the  safety  of  flight  during  the  early 
learning  stages.  The  early  flight  simulators  of  the  1930's  and  1940's 
were  crude  and  low  in  fidelity  by  present-day  standards,  but  were  a  help 
nonetheless.  The  war  effort  brought  increased  emphasis  on  the  quality 
and  availability  of  simulators,  but  their  expense  also  increased  dramati¬ 
cally.  In  the  1960's,  flight  simulators  started  to  utilize  analog  and  digital 
computers  for  flight  equations,  handling  qualities,  motion  systems,  and 
visual  scene  generation  (Koonce,  1976).  However,  they  became  too 
expensive  for  operational  use  in  selection  of  flight  training  candidates 
(Hopkins,  1975). 

Since  the  1970's  the  Air  Force  has  conducted  research  on  the  use 
of  computers  in  the  selection  of  flight  training  applicants.  This  work  has 
taken  two  approaches.  One  approach  was  to  use  performance  measures 
recorded  on  examinees  operating  a  flight  simulator  in  a  type  of  job 
sample  test.  One  early  project  using  GAT-1  flight  trainers  and  a  Varian 
computer  was  designed  to  measure  the  applicant's  performance  as  he 
was  taught  basic  flight  maneuvers  and  how  to  fly  a  basic  traffic  pattern 
(Long  &  Varney,  1975).  This  system,  the  Automated  Pilot  Aptitude 
Measurement  System  (APAMS),  was  intended  to  reduce  the  cost  of  pilot 
training  by  eliminating  the  Flight  Instruction  Program  that  used  real 
aircraft.  The  system  was  expensive  and  measured  over  170  variables. 
The  resulting  regression  equation  was  massive  and  difficult  to  cross- 
validate. 

One  predictor  measured  by  APAMS,  two-axis  tracking  skill,  has 
subsequently  been  shown  to  account  for  a  large  part  of  the  variance  in 
pilot  training  performance  as  indicated  by  pass/fail  and  overall  class 
standing.  This  relationship  has  been  found  in  research  at  US  Air  Force 
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Academy  (Koonce,  1978),  the  Air  Force  Human  Resources  Laboratory 
(Bordelon  &  Kantor,  1986),  and  the  Naval  Aerospace  Medical  Research 
Laboratory  (Griffin  &  Koonce,  1991). 

The  second  approach  applied  to  computerized  measurement  in 
pilot  selection  uses  the  computer  to  measure  basic  information-process¬ 
ing  abilities.  The  first  effort  of  this  type  In  the  Air  Force  was  by  Pew, 
Rollins,  Jager-Adams,  and  Gray  (1977).  This  group  assembled  a  battery 
of  eight  information-processing  tasks  and  a  job  sample  6onsisting  of 
measures  taken  from  a  flight  simulator.  The  resulting  regression  equa¬ 
tion  comprised  15  variables  including  the  job  sample  and  accounted  for 
about  27%  of  the  variance  in  contact  check  ride  scores.  Various  tasks 
contributed  to  the  equation.  Subsequently,  Imhoff  and  Levine  (1981) 
published  a  set  of  recommended  tasks  for  pilot  selection.  The  Basic 
Attributes  Test  (BAT)  has  evolved  from  these  early  efforts  (Carretta, 
1987).  The  BAT  effort  has  been  concerned  with  developing  a  computer- 
based  battery  of  basic  cognitive,  perceptual,  and  psychomotor  tasks,  and 
personality  measures  that  supplement  to  the  Air  Force  Officer  Qualifying 
Test  (AFOOT)  in  predicting  undergraduate  pilot  training  (DPT)  outcomes. 
Carretta  (1989)  reported  that  a  model  which  combines  AFOOT  and  BAT 
variables  accounted  for  19%  of  the  variance  in  DPT  pass/fail. 

Perhaps  the  best  way  to  view  BFITS  is  as  a  third  approach  to  using 
computerized  measurement  in  pilot  selection.  BFITS  was  not  designed  to 
measure  basic  abilities,  nor  was  it  designed  to  measure  flying  aptitude 
with  a  type  of  job-sample  approach.  Instead,  BFITS  was  designed  to  give 
a  detailed  picture  of  the  process  by  which  naive  learners  acquire  the  skill 
of  flying  a  simulated  general  aviation  aircraft.  In  this  case,  the  depiction 
of  the  skill  acquisition  process  would  need  to  be  quantitative.  A  key 
component  in  the  conceptual  design  of  BFITS  was  the  selection  of  per¬ 
formance  measures  that  should  be  recorded  during  instruction  and 
practice.  These  performance  measures  would  be  used  as  dependent 
variables  in  subsequent  validation  studies  of  basic  cognitive  ability  tests 
developed  by  LAMP. 

THE  DESIGN  AND  DEVELOPMENT  OF  BFITS 
Overview  of  BFITS 

BFITS  design  was  driven  in  part  by  recent  developments  in  cogni¬ 
tive  psychology  concerning  the  acquisition  of  skill.  The  acquisition  of 
most  complex  skills  typically  proceeds  through  three  stages  (Anderson, 
1983):  a  declarative  stage  in  which  the  learner  acquires  the  basic  con¬ 
cepts  and  principles  underlying  skilled  performance;  a  procedural  stage 
in  which  the  declarative  knowledge  becomes  compiled  into  a  more  easily 
applied  form  and  the  errors  in  the  initial  understanding  of  the  task  are 
gradually  detected  and  then  eliminated  through  practice;  and  an  auton¬ 
omous  stage  in  which  task  performance  gradually  becomes  more  refined, 
faster,  and  less  attention-demanding  with  continued  practice.  In  keeping 
with  this  view  of  skill  development,  a  basic  flight  procedures  tutor  should 
teach  a  subset  of  the  basic  concepts  and  principles  of  flying  (declarative 
stage)  and  then  provide  the  opportunity  to  practice  the  application  of 
these  in  the  flight  simulator  component  of  the  tutor  (procedural  stage). 
The  tutor  should  be  designed  to  assess  the  learner's  knowledge  and  skill 
at  several  points  in  the  learning  process. 
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In  what  follows,  the  three  components  or  modules  comprising 
BFITS  are  briefly  described.  Subsequent  sections  will  describe  these  in 
more  detail. 

The  Instructor  Module 

The  instructor’s  main  purpose  is  to  manage  the  process  of  teaching 
the  declarative  knowledge  underlying  basic  flight.  It  presents  the  student 
with  iogical  blocks  of  information.  The  content  of  the  blocks  can  be  any 
combination  of  text,  graphics,  and  animation.  The  instructor  also  han¬ 
dles  testing,  review  of  information,  and  student  progress  through  the 
tutorial.  Various  options  such  as  back-paging  through  a  lesson,  review¬ 
ing  previously  completed  lessons,  and  allowing  the  student  to  quit  in  the 
middle  of  a  lesson  are  configurable. 

The  Simulator  Module 

The  BFITS  simulator  is  designed  to  model  the  flight  characteristics  of  a 
small  single-engine  airplane.  It  teaches  the  how  of  basic  flight,  and  can 
be  used  to  demonstrate  and  practice  a  number  of  flight  maneuvers  from 
simple  climbs  and  descents  to  more  complex  maneuvers  such  as  cross- 
wind  landings.  The  simulator  is  controlled  with  a  joystick  and  rudder 
pedals.  These  are  connected  to  a  game  card  installed  in  an  IBM  or 
compatible  PC.  The  simulator  provides  a  graphical  representation  of  the 
airplane's  control  panel,  as  well  as  an  out-the-window  view.  An  EGA 
video  card  and  a  math  coprocessor  are  required.  The  simulator  works  in 
conjunction  with  the  performance  evaluator  module. 

The  Performance  Evaluator  Module 

The  performance  evaluator  works  In  concert  with  the  flight  simula¬ 
tor.  It  is  responsible  for  establishing  a  flight's  initial  conditions,  such  as 
altitude  and  airspeed.  It  tracks  the  student's  performance  relative  to 
predefined  criteria,  and  lets  the  student  know  (in  the  form  of  hint  mes¬ 
sages)  how  the  current  flight  measures  up  to  the  established  criteria. 
Once  a  flight  has  been  completed,  the  evaluator  determines  the  student's 
pass/fail  status,  and  then  gives  the  student  the  option  of  viewing  a 
graphical  display  of  his/her  performance.  The  student  also  has  the 
option  of  replaying  the  flight.  During  a  flight  replay,  the  evaluator  lets 
the  student  know  when  the  pass/fail  criteria  have  been  exceeded.  Final¬ 
ly,  the  performance  evaluator  has  the  job  of  saving  to  disk  all  data  rele¬ 
vant  to  a  flight.  These  data  can  be  used  to  replay  the  student's  flight(s), 
and  can  be  imported  into  a  statistical  package  for  detailed  analysis. 

Research  Basis  of  Performance  Criteria 

Performance  measurement  in  BFITS  is  based  on  the  principles  of 
criterion-referenced  testing  (Nitko,  1980).  This  means  that  the  students 
continue  to  perform  the  task  until  attaining  the  desired  level  of  proficien¬ 
cy.  The  criteria  to  which  a  students'  performance  is  referenced  can  be 
determined  empirically  or  taken  from  established  sources. 
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In  the  BFITS  program,  most  of  the  criteria  were  taKen  from  the 
Federal  Aviation  Administration  (FAA)  Practical  Test  Standards  (PTS)  for 
pilots  seeking  a  single-engine  airplane  pilot's  certificate  (FAA,  1988).  In 
the  PTS,  the  FAA  also  states  tolerances  for  desired  performance  of  specif¬ 
ic  flight  parameters  in  the  various  maneuvers.  Some  variables  that 
BFITS  monitors  and  scores  are  not  specified  in  the  PTS,  but  are  taken 
from  prior  research  by  Koonce  (1979,  1987).  The  tolerances  used  by 
Koonce,  Gold,  &  Moroze  (1986)  were  developed  from  the  data  of  over  300 
Air  Force  Academy  Cadets  performing  basic  flight  maneuvers  and  then 
taking  50%  of  the  magnitude  of  the  standard  deviation  of  the  perform¬ 
ance  errors  on  each  of  the  monitored  flight  variables  as  the  tolerance 
limits. 


In  the  performance  of  turns,  climbs,  and  other  dynamic  maneu¬ 
vers,  some  of  the  criterion  variables  that  are  monitored  and  subsequently 
evaluated  may  not  be  constant  throughout.  For  example,  to  begin  a 
turn,  the  wings  should  be  level  and  the  heading  stable.  When  the  turn  is 
sta  ted,  there  is  a  period  of  time  in  which  the  bank  angle  is  transitioning 
from  zero  to  the  desired  bank  angle,  such  as  30  degrees.  During  the 
transition  into  a  turn,  the  bank  angle  is  not  scored.  But  after  reaching  a 
certain  point  the  bank  angle  should  be  established  and  evaluated.  A 
logic  system  had  to  be  developed  to  determine  just  when  to  score  and  not 
to  score  these  variables  that  are  supposed  to  have  constant  values  at 
some  times  and  not  at  other  times. 

The  transition  logic  used  by  BFITS  was  determined  by  a  margin  of 
at  least  50%  over  the  general  rules-of-thumb  used  in  the  performance  of 
the  maneuvers.  For  example,  the  rules  or  guidelines  for  rolling  out  of 
turns  is  that  they  should  be  started  at  approximately  one-half  of  the 
bank  angle's  degrees  prior  to  the  desired  heading.  There  are  similar 
rules-of  -thumb  for  other  flight  maneuvers. 

The  number  of  trials  that  it  takes  a  subject  to  perform  specific 
tasks  within  tolerances  to  preset  criteria  has  been  found  to  be  a  rather 
stable  measure  of  performance  (Koonce,  1978)  and  useful  in  the  predic¬ 
tion  of  success  in  training  systems  that  are  heavily  dependent  on  stu¬ 
dents  acquiring  certain  skills  within  fixed  time  constraints  (Griffin  and 
Koonce,  1991).  in  military  flight  training  environments,  student  pilots 
are  expected  to  complete  various  phases  of  their  flight  training  within  set 
numbers  of  flight  hours.  Failure  to  do  so  results  in  a  review  flight  and 
possible  termination  from  the  program.  Thus,  one  can  ray  that  the  rate 
of  acquisition  of  the  flight  training  skills  might  be  a  criterion  that  pilot 
selection  programs  are  really  trying  to  predict. 

As  a  research  instrument,  BFITS  provides  the  researcher  with  the 
numbers  of  trials  that  the  students  needed  to  achieve  the  criterion  level 
of  performance  on  each  flight  task.  Also,  in  using  trials  to  criterion,  one 
can  be  confident  that  subjects  are  at  similar  levels  of  skill  as  they 
progress  from  basic  flight  tasks  to  advanced  flight  tasks  because  one 
must  demonstrate  a  certain  skill  level  before  progressing  to  the  next  level 
of  difficulty. 
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Instructor  and  Performance  Evaluator  Design 

Sytlabus  D.evfitQpmeD.t 

A  Basic  Flight  Instruction  Syllabus  was  developed  covering  the 
fundamental  skills  required  to  pilot  an  airplane.  This  syllabus  is  the 
result  of  an  analysis  of  the  training  requirements  appropriate  to  fulfill  the 
primary  and  secondary  BFITS  requirements,  which  are  the  creation  of 
research  data  and  the  instruction  of  students.  It  was  assumed  that  the 
students  working  with  BFITS  would  have  the  ba*:ic  intellectual  and 
perceptual-motor  skills  required  for  acceptance  to  a  pilot  training  pro¬ 
gram:  a  high  school  education,  a  desire  to  fly  airplanes,  and  no  percep¬ 
tual-motor  difficulties.  The  students  may  be  completely  naive  regarding 
airplanes  and  the  theory  of  flight. 

The  topics  covered  by  BFITS  are  a  subset  of  those  typically  covered 
In  flight  training  courses  as  described  by  the  FAA  Flight  Training  Hand¬ 
book  (1980),  the  Private  Pilot  Trairiino  Syllabus  (Jeppesen-Sanderson, 
1988),  and  the  University  of  Illinois'  Institute  of  Aviation  Basic  Flight 
Instruction  Trainjng  Syllabus  (1989).  The  topics  begin  with  fundamental 
concepts  and  build  up  to  more  complex  topics.  Ground  school  and  flight 
training  programs  in  general  proceed  in  this  manner  so  that  the  student 
always  has  some  foundation  for  the  next  topic  in  the  sequence.  With  this 
building  blocks  approach  "...  a  student  can  master  the  segments  of  the 
overall  pilot  performance  requirements  individually  and  can  progressively 
combine  these  with  other  related  segments  until  their  sum  meets  the 
final  objective."  (FAA,  1977,  AC  60-14,  p.  78).  This  is  the  general  ap¬ 
proach  that  wac  taken  in  determining  the  sequence  of  the  lesson  materi¬ 
als  and  skills  deveio:>ment  required  by  BFITS. 

The  syllabus  is  based  upon  a  careful  analysis  of  the  knowledge  and 
skills  required  for  the  safe  flying  of  an  airplane.  The  lesson  objectives  are 
oriented  toward  the  development  of  knowledge  (declarative  stage),  in¬ 
creased  understanding  of  knowledge  through  application  (procedural 
stage),  translation  of  knowledge  to  perceptual-motor  tasks  (procedural 
stage)  and  development  of  skills  to  preset  criterion  levels  (autonomous 
stage).  The  lesson  materials  are  sequenced  from  the  most  basic  and 
fundamental  concepts  (building  blocks)  to  the  more  complex  ones  that 
are  dependent  upon  the  earlier  materials. 

The  BFITS  syllabus  aims  to  teach  a  subject  the  basic  principles  of 
flying  an  airplane.  This  inciudes  presentations  on  the  principles  of  flight, 
the  parts  of  an  airplane,  and  how  the  airplane  is  controlled  in  flight. 
Following  the  development  of  basic  declarative  knowledge,  the  student  is 
taught  how  to  fly  the  airplane  (simulator).  The  flight  instruction  portion 
includes  knowledge  development  regarding  the  maneuvers  to  be  per¬ 
formed  in  the  current  lesson  as  well  as  instruction  and  a  demonstration 
on  how  to  perform  the  maneuvers.  The  flight  instruction  proceeds  from 
very  basic  maneuvers  through  the  complex  task  of  performing  takeoffs 
and  landings  in  the  presence  of  crosswinds. 

BFITS  proceeds  from  basic  flying  skills  through  the  performance  of 
traffic  patterns  (takeoff  to  landing).  Many  topics  that  are  not  germane  to 
these  goals,  although  usually  taught  in  flight  schoois,  are  not  included  in 
the  BFITS  syllabus.  These  include,  but  are  not  limited  to:  Required 
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Certificates  &  Documents,  Weight  &  Balance,  C!^  acklist  Usage,  Preflight 
Inspection,  Detailed  Engine  Operations,  Radio  Communications,  Emer¬ 
gency  Procedures,  Cross-Country  Procedures  and  Techniques,  Night 
Flying,  and  Meteorology.  Some  of  these  items  couid  easily  be  incorporat¬ 
ed  into  the  syllabus  but  are  not  directly  relevant  to  the  BFITS  research 
mission.  Of  course,  a  complete  pilot  training  program  would  include  all 
of  the  above  plus  physiological  factors  associated  with  flight  and  other 
topics. 

Some  of  the  topics  covered  by  BFITS  can  be  approached  in  several 
different  ways.  The  approach  is  generally  determined  by  the  individual 
instructor,  his  or  her  own  conceptions,  and  experience  in  teaching 
students  about  flying.  The  order  of  topics  in  BFITS  was  based  on  our 
subject  matter  expert's  (SME’s)  experience  and  consideration  of  materials 
published  by  the  FAA  (AC  60-14,  AC  6 1-21  A,  AC  61-23B,  and  the  practi¬ 
cal  test  standards  for  private  pilot  certification),  training  course  outlines 
(TCOs)  approved  by  the  FAA  for  flight  schools,  several  flight  instruction 
textbooks,  and  the  materials  of  several  companies  who  distribute  flight 
training  materials. 

The  initial  syllabus  consisted  of  three  lessons  concerned  with  the 
development  of  knowledge  (declarative  stage)  and  nineteen  lessons  involv¬ 
ing  the  development  of  both  knowledge  and  flying  skills.  As  Phase  II  of 
the  effort  progressed,  we  realized  that  the  first  three  lessons  would  be  too 
demanding  on  the  students  having  no  prior  knowledge  of  flying;  so,  the 
three  lessons  were  divided  into  nine  lessons.  The  flight  lessons  were 
increased  by  three,  from  nineteen  to  twenty-two. 

The  first  nine  lessons  develop  the  concepts  of  how  airplanes  fly,  the 
parts  of  an  airplane,  and  what  the  pilot  needs  to  do  to  fly  an  airplane. 
The  remaining  lessons  present  demonstrations  of  maneuvers  to  students 
and  provide  guided  practice  opportunities  (see  Table  1). 

The  BFITS  lesson  plan  is  appropriate  to  BFITS  simulation  capabili¬ 
ties  while  covering  a  large  portion  of  the  entire  basic  flight  training  spec¬ 
trum.  More  importantly,  it  will  create  a  wide  range  of  data  sets  for  use  in 
research,  varying  from  simple  response  time  arrays  to  multiple  variable 
sets  acquired  during  the  execution  of  complex  multiple  task-loaded 
procedures. 

Instructional  Methodology 

Our  selection  of  training  methodologies  was  influenced  by  two 
factors,  training  effectiveness  and  suitability  for  research  purposes.  The 
training  goal  of  BFITS  is  to  develop  the  individual's  skill  in  flying  the 
BFITS  simulator  through  a  required  scenario  of  maneuvers,  and  to 
maximize  the  transfer  of  skills  to  a  real  training  aircraft.  The  research 
goal  of  BFITS  is  to  collect  useful  and  accurate  data  relating  to  the  acqui¬ 
sition  of  flight  skills. 

The  approach  taken  in  the  development  of  this  syllabus  and  in  the 
planning  of  the  instructional  and  evaluatiori  techniques  involved  integra¬ 
tion  of  principles  and  techniques  expressed  in  FAA's  Aviation  Instructor's 
Handbook  (AC  60-14,  1977),  the  U.  S.  Air  Force's  Principles  aoii  Tech¬ 
niques  sif  Instruction  (AFM  50-9),  Principles  sd  lnslr.Ufitj.Qn.al  Besign  by 
Gagne  and  Briggs  (1979),  the  Aircraft  Owners  and  Pilots  Association's 
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Table  1.  BFITS  Syllabus 


Knowledge  (Declarative  Stage) 

1  introduction  and  principles  of  lift  and  airfoils 

2  Anatomy  of  an  airplane 

3  The  axes  of  flight  and  movement  about  them 

4  Airplane  control  surfaces 

5  Other  controls 

6  Basic  four  forces  acting  upon  the  airplane 

7  Other  forces  acting  on  the  airplane 

8  Cockpit  displays 

9  Response  of  the  instruments  to  the  control  inputs  and  the 
resulting  effects  on  the  airplane 

Flight  Skills  (Procedural  Stage) 

Block  I  (Basic  Flight  Procedures): 

10  Climbs,  descents,  leveloffs,  and  cruise 

1 1  Shallow  and  medium  banked  turns 

1 2  Climbing  and  descending  turns 

1 3  Airspeed  transitions 

1 4  Review  test  lesson 

Block  II  (Stall-Related  Maneuvers): 

1 5  Introduction  to  slow  flight  (no  flight  tasks) 

1 6  Slow  flight  and  stall  recovery 

1 7  Landing  and  takeoff  stalls 

1 8  Steep  turns 

19  Accelerated  stalls 

20  Review  test  lesson 

Block  III  (Ground  Referenced  Maneuvers): 

21  Rectangular  patterns  with  and  without  winds 

22  Turns  about  a  point,  with  and  without  wind 

23  Introduction  to  traffic  patterns  (no  flight  tasks) 

24  Takeoff,  upwind,  cross  wind,  &  turn  downwind 

25  Downwind,  base,  and  final 

26  Flare  and  landings  -  no  wind 

27  Go-around 

28  Complete  traffic  pattern  -  no  wind 

29  Crosswind  takeoff  and  landing 

30  Complete  traffic  pattern  with  crosswind 

31  Review  test  lesson 


Flight  Instructor's  Flioht/Ground  Instructor  Course  (undated),  and  the 
Basic  Flight  Training  Syllabus  (University  of  Illinois  Institute  of  Aviation, 
1989). 

The  format  used  for  the  first  nine  lessc.is  is  to  give  (a)  a  brief  over¬ 
view  of  the  current  lesson's  topic,  (b)  a  review  of  the  material  presented  in 
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the  previous  lesson(s)  that  would  be  useful  in  the  current  lesson,  (c) 
presentation  of  the  lesson  with  review  questions  on  each  major  topic 
covered,  (d)  a  summary  of  the  lesson  Just  covered  and  a  brief  overview  of 
the  topicis)  to  be  presented  in  the  next  lesson,  and  (e)  a  test  covering  the 
entire  lesson  and  feedback  on  the  test  results. 

Beginning  with  Lesson  10,  BFITS  presents  a  demonstration  of  the 
maneuvers  that  the  student  is  to  perform  for  that  lesson.  This  occurs 
immediately  after  the  comprehension  test  and  feedback.  The  student 
then  practices  the  maneuvers  until  reaching  the  criterion  level  of  per¬ 
formance. 

Three  of  the  lessons,  14,  20  and  31,  are  simulator  flight  tests  in 
which  the  students  are  asked  to  perform  certain  maneuvers  that  had 
been  learned  to  a  criterion  level  of  performance.  These  flight  tests 
measure  retention  of  previously  learned  flight  simulator  skills. 

Lessons  follow  a  pattern  (see  Figure  1).  Each  begins  with  a  brief 
overview  of  the  current  lesson.  This  is  followed  by  a  review  of  concepts 
and  skills  relevant  to  the  current  task  that  were  learned  in  the  previous 
lesson.  When  the  student  finishes  the  review,  the  lesson  presentation 
begins.  Lesson  presentation  is  structured  such  that  the  student  is  intro¬ 
duced  to  new  material  in  logical  units,  or  blocks  of  frames.  Most  lessons 
consist  of  several  blocks,  so,  BFITS  presents  a  brief  question  after  each 
block  of  information.  The  question  samples  the  student's  knowledge  of 
the  material  just  covered.  If  the  student  responds  correctly,  the  program 
presents  the  next  lesson  block.  If  the  question  is  answered  incorrectly, 
BFITS  repeats  the  relevant  frames,  highlighting  (in  yellow  typeface)  perti¬ 
nent  information  and  skipping  any  areas  which  do  not  apply  to  the 
question.  If  the  question  is  incorrectly  answered  a  second  time,  then  the 
student  is  given  an  explanation  of  why  the  answers  were  not  correct. 
Likewise,  a  complete  explanation  of  the  correct  answer  is  given. 

At  the  end  of  each  lesson,  there  is  a  quiz  (typically  10  questions) 
covering  the  material  just  presented.  After  completing  a  quiz,  the  student 
is  given  a  second  opportunity  to  answer  the  incorrectly  answered  ques¬ 
tions.  The  format  followed  is  similar  to  that  at  the  end  of  the  block 
presentations. 

The  materials  are  presented  in  a  manner  that  is  both  interesting 
and  motivating.  Each  of  the  general  topical  areas  is  broken  into  smaller 
blocks  or  lessons  in  which  specific  concepts  are  presented  and  skills 
practiced  until  satisfactorily.  Each  simple  task  is  performed  satisfactorily 
before  the  next  learning  task  is  introduced.  Lessons  are  brief  to  mini¬ 
mize  the  probability  of  answering  test  items  incorrectly,  while  still  keep¬ 
ing  students  active,  involved,  and  sufficiently  challenged.  When  a  ques¬ 
tion  is  not  answered  correctly,  a  review  of  relevant  material  is  presented, 
and  if  further  difficulty  is  encountered  the  system  explains  the  reasoning 
behind  both  incorrect  and  correct  answers.  Helpful  feedback  is  provided 
to  maintain  a  positive  attitude  in  the  students,  and  to  help  reinforce  the 
student’s  memory  for  the  concept  or  fact  being  taught. 

Questions  in  the  lesson  quizzes  are  designed  to  assess  (a)  the 
student's  comprehension  and  retention  of  the  information  presented  in 
the  lesson  (discriminations  and  concrete  concepts),  and  (b)  his/her  ability 
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Figure  1 .  Lesson  Flow 

to  apply  the  concepts  learned  to  different  situations  (rule  understanding  and 
application).  These  application  questions  and  the  practice  of  the  maneuvers  may 
be  thought  of  as  opportunities  to  observe  the  student  in  near-transfer  conditions, 
success  in  which  would  reflect  a  more  sophisticated  state  of  knowledge  development 

Lessons  10  through  31  include  the  development  of  motor  skills  as 
well  as  intellectual  skills,  and  therefore  include  the  measurement  of 
perceptual-motor  ability,  including  an  assessment  of  the  rate  of  skill 
acquisition.  Learning  to  perform  the  motor  skills  involved  in  controlling 
an  airplane  entails  several  components.  In  any  given  maneuver  the 
student  must  learn  to  attend  to  the  appropriate  stimuli,  perceive  their 
significance  with  respect  to  a  goal,  note  discrepancies  between  the  stimu¬ 
li  and  the  desired  state,  and  make  the  necessary  corrections.  For  exam- 
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pie,  in  executing  a  45  degree  right  turn  at  80  miles  per  hour,  the  student 
must  know  to  attend  to  airspeed,  bank  angle,  altitude,  and  vertical 
speed.  If  the  student  notices  that  altitude  is  decreasing,  he  must  add 
power  (increase  RPM  to  2250),  and  adjust  the  elevator  trim  to  restore  the 
aircraft  to  a  level  flight  turn.  BFITS  monitors  the  appropriate  variables 
and  provides  guidance  in  the  form  of  hints  or  messages  that  warn  the 
student  when  variables  exceed  tolerances. 

Attempting  to  monitor  many  variables  and  give  the  pilot  hints 
introduced  some  problems  beyond  the  capability  of  the  performance 
monitoring  program.  For  instance,  if  the  airplane  is  at  a  desired  altitude 
and  the  vertical  speed  indicator  begins  to  show  a  climb,  the  hint  screen 
would  give  the  appropriate  message  of  "You're  climbing!  Check  your  pitch 
and  power."  But,  if  the  airplane  is  below  the  desired  altitude  the  hint 
screen  might  say  "Attitude  low.  Should  be  3000  feet."  When  the  pilot 
begins  a  climb  back  to  the  desired  altitude  the  message  "You're  climbing! 
Check  your  pitch  and  power"  is  inappropriate  because  the  pilot  is 
making  a  proper  correction  back  to  the  desired  altitude. 

An  on-board  instructor  would  know  when  the  climb  hint  is  appro¬ 
priate  and  when  it  is  not.  The  Flight  Criteria  program  is  basic  in  that  it 
does  not  have  the  logic  built  in  to  apply  the  If-Then  logic  that  is  appro¬ 
priate.  If  below  altitude,  then  climbing  is  permissible;  if  at  desired  alti¬ 
tude,  then  climbing  is  not  desired.  Similar  logic  problems  arise  in  the 
monitoring  of  flight  variables  while  the  airplane  is  following  a  desired 
ground  track. 

The  logical  inconsistencies  could  be  avoided  in  two  ways:  (a)  by 
adding  a  built-in  logic  system  or  (b)  by  monitoring  only  the  primary 
variables,  giving  hints  when  they  stray,  and  to  not  give  hints  when  the 
secondary  variables  start  to  deviate  with  the  potential  of  having  excessive 
error  in  a  primary  variable.  It  was  felt  that  the  former  was  too  cumber¬ 
some  for  the  system  as  it  was  configured.  The  latter  was  implemented 
and  has  proven  a  suitable  approach  at  this  early  stage.  This  should  be 
kept  in  mind  in  any  future  validation  studies. 

A  record  is  made  of  all  answers  given  by  the  student,  along  with 
various  flight  data  (see  Table  2).  This  record  can  be  utilized  initially  for 
the  improvement  of  BFITS  (i.e.,  checking  to  insure  that  the  materials 
were  presented  well  and  that  the  questions  were  not  vague  or 
misleading). 

This  record  should  allow  the  researcher  to  construct  a  lesson 
history  indicating  the  items  a  student  missed,  which  sections  he/she 
was  required  to  review,  how  much  time  was  spent  reviewing  these  sec¬ 
tions,  and  so  forth.  Additionally,  this  data  will  aid  in  the  formative 
evaluation  of  the  lessons  and  the  overall  program. 

Simulator  Module  Design 

The  initial  hardware  specification  dictated  that  the  runtime  system 
would  be  a  Zenith  Z-248  with  EGA,  20  megabyte  hard  drive,  math  co¬ 
processor,  640K  plus  2  megabytes  of  extended  memory.  This  has  been 
the  standard  system  in  the  LAMP  (Learning  Abilities  Measurement  Pro¬ 
gram)  laboratory  for  several  years.  The  LAMP  laboratory  is  in  the  process 
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of  acquiring  Desktop  3  systems;  these  will  be  the  standard  platform  for 
future  LAMP  and  BFITS  testing. 

The  initial  development  of  BFITS  was  performed  using  various 
systems  (Z-248,  Z-286  LP,  Compustar,  IBM  PS/2  Model  30,  and  others). 
The  relative  value  of  the  data  to  be  collected  by  the  BFITS  systems  was 
significantly  influenced  by  the  lack  of  similarity  in  computer  systems  and 
the  corresponding  variations  in  simulation  update  rate.  It  was  decided 
that  the  best  solution  was  to  standardize  the  computer  systems  being 
used.  To  provide  for  an  easy  transition  to  the  Desktop  3  systems,  five 
standardized  hardware  kits  were  acquired  which  allowed  processor 
upgrade  to  a  level  of  performance  comparable  to  that  of  the  Desktop  3. 


Table  2^  Logged  data 

Question  Response  Data: 

Response  latency. 

Correct  responses. 

Incorrect  responses. 

Incorrect  response  specifics. 
All  scores. 

Comprehensive  Practice  Flight  Data: 
Aileron  position 
Elevator  position 
Rudder  position 
Engine  RPM 
Flap  position 
Bank 
Pitch 
Heading 
Altitude 
Air  speed 
Vertical  speed 
Rate  of  turn 
Ball  position 

World  X  and  z  coordinates 
Segment  number 
Trial  number 
Step  number 


Also  acquired  for  this  effort  were  40  Simulator  Accessory  Kits. 
These  include  a  Joystick  (CH  Products'  FlightStick),  joystick  interface  (a 
standard  gamecard),  and  a  custom  rudder  pedal  assembly.  These  de¬ 
vices  allow  full  control  of  simulated  aileron  (with  trim),  rudder,  elevator 
(with  trim),  and  throttle. 

A  complete  description  of  hardware  requirements  is  included  in  the 
system  documentation.  Any  potential  system  user  should  be  aware  that 
the  BFITS  system  has  been  designed  for  operation  on  Desktop  3  (or 
equivalent)  systems.  Use  on  other  hardware  may  or  may  not  result  in 
reliable  operation  and  accurate  flight  simulation  behavior. 
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The  simulator  has  been  designed  to  replicate  the  characteristics  of 
a  generic  training  aircraft.  The  performance  characteristics  of  the  C- 
172/T-41  trainer  were  used  as  a  baseline  in  developing  simulator  char¬ 
acteristics.  The  flight  model  was  designed  in-house  after  a  search  for 
existing  flight  models  was  conducted  through  both  the  Defense  Technol¬ 
ogy  Information  Center  (DTIC)  and  the  Federal  Laboratory  Consortium 
Clearinghouse.  Table  3  lists  the  major  flight  relationships  included  in 
the  BFITS  simulation  flight  model. 

Many  aircraft  flight  dynamics  are  defined  within  a  data  table, 
allowing  changes  to  the  flight  model  to  be  made  with  a  minimum  of  ef¬ 
fort.  Most  of  the  software  has  been  developed  using  Pascal  and  industry- 
standard  modular  programming  techniques.  Some  real-time  graphics 
functions  have  been  implemented  using  assembly  language  to  increase 
system  speed  and  performance. 

External  simulator  features  include  display  of  the  "sacred  six" 
flight  instruments  (airspeed,  attitude,  altimeter,  vertical  speed,  heading, 
and  turn  and  bank  indicator),  plus  flap  indicator,  navigational  radio 
equipment,  engine  instrumentation,  and  out-the-cockpit  view.  Position 
indicators  are  provided  for  the  aileron,  elevator,  and  rudder. 


Table  2.  BFITS  flight  model  relationships 

Rudder  /  Bank  effect 
Aileron  /  Bank  effect 
Rudder  /  Yaw  effect 
Aileron  /  Yaw  effect 
Power  /  Yaw  effect 
Elevator  /  Pitch  effect 
Power  /  Pitch  effect 
Aircraft  Roll  Stability 
Pitch  Stability 
Bank  /  Rate  of  Turn  effect 
Pitch  /  Rate  of  Turn  effect 
Rudder  /  Rate  of  Turn  effect 
Engine  Thrust  Effect 
Airframe  Drag 

Angle  of  Attack  effect  on  Thrust  (Drag) 

Aircraft  Weight 

Power  contribution  to  lift 

Angle  of  Incidence 

Wing  Lift  capability  as  a  function  of  AOA 
Stalling  Angle  of  Attack 
Stall  Clean  (VS1) 

Stall  Dirty  (VSO) 

Flap  effect  on  Pitch 
Flap  effect  on  Thrust  (Drag) 

Airframe  Service  Ceiling 
Engine  Service  Ceiling 
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The  out-the-cockpit  view  has  several  unique  features.  The  stand¬ 
ard  field  of  view  is  25%  greater  than  would  normally  be  provided.  The 
size  of  the  images  in  displays  of  the  outside  world  has  been  found  to  have 
a  significant  influence  on  a  pilot's  ability  to  perform  landings  and  take¬ 
offs.  Roscoe  (1948,  1951)  and  Roscoe,  Hasler  and  Dougherty  (1966), 
using  a  projection  periscope  in  a  Cessna  T-50  airplane,  found  that  land¬ 
ings  were  most  accurate  in  terms  of  both  constant  and  variable  errors 
using  an  image  magnification  of  1.25.  Roscoe's  results  were  confirmed 
by  Campbell,  McEachern,  and  Marg  (1955)  using  a  binocular  periscope 
for  approach  and  landing.  Because  of  these  findings,  the  out-the-window 
view  in  the  BFITS  simulator  was  changed  to  have  a  magnification  of  1.25 
as  opposed  to  the  original  1 .00. 

A  second  magnification  level  of  3.00  has  also  been  provided.  Since 
there  are  no  "side  windows"  available,  this  method  allows  earlier  detec¬ 
tion  of  the  runway  and  airport  environment  when  flying  traffic  patterns. 
Toggling  between  the  standard  magnification  (1.25)  and  the  wide-angle 
magnification  (3.00)  is  performed  with  a  press  of  the  "V"  (for  view)  key. 
The  outline  of  the  standard  view  perimeter  is  provided  when  using  the 
wide-angle  mode  as  an  indication  that  the  mode  is  active  and  to  assist  in 
deciding  when  to  switch  back  to  standard  mode. 

An  overhead  view  may  also  be  superimposed  over  the  right  side  of 
the  out-the-cockpit  view.  This  provides  a  simple  means  of  maintaining 
orientation  to  the  runway  when  it  is  not  within  either  field  of  view.  It 
allows  the  pilot  to  maintain  runway  /  aircraft  orientation  while  circling 
the  airport  traffic  pattern.  It  also  assists  in  the  instruction  of  cross  wind 
correction  techniques  and  ground  tracking.  A  similar  overhead  view  is 
provided  for  use  in  learning  turns  around  a  point. 

The  actual  simulator  environment  includes  the  runway,  buildings, 
roads,  mountains,  and  trees.  The  runway  includes  Visual  Approach 
Slope  Indicator  (VASI)  lighting  and  centerline  markings.  Roads  and  ter¬ 
rain  features  can  be  used  in  learning  crosswind  correction  techniques 
and  as  external  heading  references.  Trees  and  buildings  provide  both 
lateral  motion  cues  along  with  height  information  essential  to  proper 
landing  techniques.  A  moderate  density  of  simple  objects  in  the  visual 
field  has  been  achieved  in  lieu  of  high  fidelity.  Kleiss,  Hubbard,  and 
Curry  (1989)  demonstrated  that  this  level  of  fidelity  to  be  effective  in 
flight  training  tasks. 

The  design  of  the  simulator  was  guided  by  the  need  to  track, 
respond  to,  and  evaluate  student  performance  while  flying  against  speci¬ 
fied  levels  of  criteria.  A  general  flight  format  was  developed  as  shown  in 
Table  4. 

Segments  can  be  thought  of  as  frames.  By  associating  a  particular 
goal  (Climb,  Turn,  Landing,  etc)  with  a  frame,  an  instructor  can  quickly 
create  a  new  flight  task  (such  as  a  climbing  left-hand  turn  in  slow  flight) 
by  simply  combining  a  number  of  segments.  The  instructor  does  this  by 
using  an  editor.  The  editor  was  developed  to  allow  creation  and  easy 
modification  of  the  flight  criteria.  There  are  context-sensitive  help  facili¬ 
ties,  multi-windowing,  error  messages,  automatic  dependency  checking, 
and  a  host  of  copy,  delete,  and  move  options.  Any  changes  to  existing 
flight  criteria  are  detected,  and  the  user  is  asked  if  he/she  wants  to 
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Table  ^  Flight  Format 

A  statement  of  the  flight/lesson  objective 
A  demonstration  by  example  of  the  desired  performance 
Guided  practice  (real-time  feedback) 

A  review  of  each  maneuver  noting  criteria  being  evaluated  and 
actual  student  performance. 

Continuation  of  practice  until  the  criterion  level  of  performance  is 
attained. 


The  term  flight  criteria  is  used  to  denote  the  block  of  information 
that  is  passed  to  the  simulator,  and  in  essence,  defines  the  student  flight 
task.  The  flight  criteria  are  arranged  in  a  hierarchy  (see  Figure  2).  Each 
level  in  the  hierarchy  is  more  fine  grained  than  the  level  above.  This 
structuring  allows  for  rapid  task  setup  and  modification,  automatic  task 
control,  and  real-time  student  evaluation  with  minimal  overhead.  A 
flight  test  may  consist  of  a  number  of  tasks.  For  example,  the  flight  test 
in  lesson  10  involves  a  climb,  straight-and-level  flight,  and  a  descent. 
Each  of  these  tasks  is  associated  with  a  segment.  Thus  in  lesson  10 
there  are  3  segments.  Tasks  are  divided  into  sub-tasks,  each  corre¬ 
sponding  to  a  step.  The  climb  segment  in  lesson  10  is  made  up  of  5 
steps.  Associated  with  each  step  is  a  group  of  variables  that  define  the 
sub-task  and  trigger  sequential  transitions  from  one  task  to  another. 
The  variables  are  also  used  to  in  student  flight  performance  evaluation. 


Figure  2.  Flight  Criteria  Hierarchy 

create  a  new  version.  By  responding  yes,  a  new  version  is  automatically  creat¬ 
ed  and  the  previous  version  is  saved  to  disk. 

The  simulation  begins  with  an  explanation  of  the  flight  objectives 
and  a  demonstration  flight.  Pressing  one  of  the  joystick  buttons  initiates 


the  student  test  flight.  During  the  flight,  the  student’s  progress  relative 
to  the  stated  objective  is  constantly  monitored.  If  the  student  begins  to 
deviate  from  the  expected  flight  parameters,  he/she  receives  hints  rele¬ 
vant  to  the  area  of  deviation.  For  example,  if  the  altitude  must  be  main¬ 
tained  and  the  student  begins  to  climb,  a  message  would  appear  on  the 
screen  saying  something  like  "You’re  starting  to  climb.  Level  off."  At  the 
same  time,  the  student’s  performance  is  being  evaluated  on  a  pass  /  fail 
basis.  At  the  end  of  a  flight,  a  student  can  look  at  a  statistical  review  of 
the  flight,  (see  Figure  3).  If  the  student  desires  even  more  information 
regarding  his/her  performance  on  the  prior  maneuver,  a  playback  of  the 
last  performance  can  be  selected.  This  will  present  the  entire  maneuver 
as  the  student  flew  it,  and  the  hint  window  will  indicate  when  the  desired 
parameter  limits  were  exceeded.  So,  not  only  does  the  student  see  the 
maneuver  that  was  just  flown,  but  is  told  when  and  where  the  limits  of 
desired  performance  were  exceeded. 


Variable:  Heading  in  degrees 

Step:  Level  off 

Segment:  Climb 

Target 

High  Pass  Limit 

Low  Pass  Limit 

300.00 

310.00 

290.00 

Average 

High 

Low 

292.54 

302.5 

288.00 

High  Pass  Limit 


Esc-exit  graph  Arrow  keys-view  a  new  variable _ Trial#:  2 


Figure  3.  Review  Screen 
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The  student  is  not  allowed  to  continue  with  later  flight  tasks  until  all  preceding 
flight  tasks  have  been  successfully  completed.  (See  Figure  4). 


Figure  4.  Simulator  Flow 


A  key  concept  in  BFITS  task  control  is  that  of  a  gate  set.  This  is 
the  mechanism  that  moves  the  simulated  flight  task  from  beginning  to 
completion.  The  gate  set  consists  of  a  gate,  evaluation  criteria,  and  an 
implicit  transition.  The  gate  is  a  value  describing  a  discrete  state  for  a 
particular  flight  variable  (see  Table  5). 


Table  Flight  Variables 

1 .  Bank 

2.  Heading 

3.  Altitude 

4.  Vertical  Speed 

5.  Air  Speed 

6.  RPM 

7.  Pitch 

8.  Rate  of  Turn 

9.  Ball  Position 

10.  Position  East-West 

1 1 .  Position  North-South 

12.  Time 

13.  Flaps 
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Figure  5.  Flight  Criteria  for  Segment  1,  Lesson  10 
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CONCLUSIONS 


1.  BFITS  is  the  first  known  PC-based  flight  simulator  system  to 
train  subjects  to  perform  such  a  wide  variety  of  maneu’  -^rs  fr^m  basic  to 
traffic  patterns  with  crosswinds,  and  require  a  minimum  level  of  per¬ 
formance  for  progression  and  successful  completion.  BFITS  can  be 
easily  modified  to  change  the  variables  monitored  and  the  tolerances  of 
acceptable  performance.  New  maneuvers  can  be  developed,  if  required, 
using  an  experienced  flight  instructor  to  develop  adequate  flight  criteria 
for  the  evaluation  of  performance. 

2.  If  different  populations  are  used,  the  criteria  can  be  changed.  In 
fact,  many  versions  of  the  flight  criteria  can  be  developed,  with  the  re¬ 
searchers  or  instructors  selecting  the  appropriate  version  of  the  criteria 
for  the  individual  subject  using  the  system  at  that  time. 

3.  BFITS  was  not  intended  to  be  the  ultimate  computer-based 
flight  instructor,  but  it  is  comprehensive  and  has  face  validity  with 
regard  to  potential  transfer  of  training  to  actual  aircraft  performance.  Its 
predictive  validity  In  that  area  must  be  determined. 


RECOMMENDATIONS 

Although  BFITS  has  been  reviewed  by  many  persons  during  its 
development,  it  is  likely  that  its  instructional  effectiveness  can  be  im¬ 
proved  through  further  research.  For  example,  the  content  of  BFITS 
lends  itself  well  to  different  types  of  illustrations  (Mayer  &  Gallini,  1990) 
and  models  (Mayer,  1989).  Further  research  could  investigate  the  effects 
of  different  types  of  illustrations  and  models  on  declarative  learning. 
Flying  skill  development  on  BFITS  might  also  be  enhanced  with  digitized 
voice  technology,  and  diagnostic  feedback  after  each  flight  session.  We 
are  concerned  that  the  on-line  hints  now  presented  in  printed  form  on 
the  screen,  may  put  the  less-skilled  reader  at  a  disadvantage.  Digitized 
voice  technology  could  be  used  to  present  these  hints,  perhaps  more 
effectively  for  the  less-skilled  reader. 

The  second  enhancement,  diagnostic  feedback,  might  require  a 
more  substantial  research  and  development  effort.  The  basic  idea  is  to 
tell  student  pilots  after  each  flight  what  they  have  done  incorrectly  and 
how  they  might  improve  their  performance.  Student  flight  data  might  be 
compared  to  prototypical  mistakes  made  by  student  pilots.  If  a  stu¬ 
dent's  data  matches  a  prototype  sufficiently  well,  the  computer  might 
then  provide  guidance  about  what  to  do  differently  when  performing  the 
maneuver.  Diagnostic  feedback  requires  prototypical  mistakes,  a  means 
of  collecting  and  representing  them,  and  a  pattern-matching  algorithm 
for  diagnosis.  Expert  flight  instructors  could  be  asked  to  simulate 
common  student  errors  on  a  given  maneuver  using  BFITS.  Developing  a 
pattern-matching  algorithm  would  be  a  significant  technical  challenge, 
but  one  possible  approach  would  be  to  compare  the  student's  profile  of 
flight  criteria  to  the  mean  profile  of  flight  criteria  obtained  from  the  flight 
instructors. 

Although  the  flight  criteria  used  by  BFITS  are  similar  to  those  used 
by  the  FAA  in  the  administration  of  the  practical  flight  tests  for  the  pri- 
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vate  pilot  certification,  the  adequacy  of  the  flight  criteria  for  a  population 
of  flight-naive  students  using  BFITS  remains  to  be  determined.  After  all, 
the  BFITS  flight  simulator  is  not  quite  the  same  as  an  airplane. 

After  a  trial  run  with  a  small  sample,  BFITS  should  be  adminis¬ 
tered  to  a  significant  number  of  flight-naive  subjects  to  obtain  normative 
data  on  the  test  items  and  flight  skills  development.  Where  possible, 
measures  of  the  transfer-of-training  capability  of  BFITS  into  a  flight 
training  program  should  also  be  obtained. 

The  first  generation  BFITS  should  be  an  effective  research  tool  for 
investigating  the  acquisition  of  flying  skills.  We  see  additional  applica¬ 
tions  for  BFITS  In  assessing  the  effects  of  drugs,  stress,  and  other  envi¬ 
ronmental  factors  on  pilot  performance. 
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